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Abstract. Reweighted ^-algorithms have attracted a lot of attention in the field 
of applied mathematics. A unified framework of such algorithms has been recently 
proposed in [34j. In this paper we construct a few new examples of reweighted l\- 
methods. These functions are certain concave approximations of the Zo-norm function. 
We focus on the numerical comparison between some new and existing reweighted 
/i-algorithms. We show how the change of parameters in reweighted algorithms may 
affect the performance of the algorithms for finding the solution of the cardinality 
iy~j ■ minimization problem. In our experiments, the problem data were generated ac- 

cording to different statistical distributions, and we test the algorithms on different 
sparsity level of the solution of the problem. Our numerical results demonstrate that 



the reweighted /i-method is one of the efficient methods for locating the solution of 



the cardinality minimization problem. 
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1 Introduction 

The cardinality minimization problem over a convex set C C lR n can be written as 

Minimize II x II n 

ii no 

s.t. x e C. 

This problem is to minimize the number of non-zero components of a vector satisfying 
certain constraints. In other words, cardinality minimization problem is looking for 
the sparsest vector in a given feasible set. 

In this paper, we suppose that C is defined by an undetermined system of linear 
equations, i.e, 

C = {x; Ax = b}, where A e R mxn (m < n) ,b e R m . 

These linear systems have infinite many solutions, and the purpose of cardinality min- 
imization problem(CMP) is to find the sparsest one, which can be stated as follows: 

Minimize llxlln 

ii no (2) 

s.t. Ax = b. 

The problem ()2|) is closely related to compressed sensing which is dealing with 
the reconstruction of sparse signals from a limited number of linear measurements 
[TT| \TT\ [29| IT5] . Also problems with cardinality constraints have a wide range of ap- 
plications, especially in portfolio optimization problems [281 HO] , and principal com- 
ponent analysis and model reduction [251 US] ■ The more generalized version of car- 
dinality minimization problems is so called rank minimization problems which have 
been considered in recent years [201 132] - 

The / -norm function is discontinuous, so the main idea for solving the problem 
fl2]) is to approximate the / - n orm function by some other continuous functions which 
are easier to deal with. For example / p -norm function(0 < p < 1) is one of the 
approximations of the Z - norm - lp minimization (0 < p < 1) has been studied in 
[271 1131 [H]. Figure (1) represents the graph of Y^t=i \ x i\ V i for p = 1, p = 0.6, and 
p = 0.2. Note that as p goes to zero, XT=i \ x i\ P approaches to the Z -norm function. As 
seen in the Figure (1), the closest convex approximation of ||x||o is the well known li- 
norm function. So it is unavoidable to use non-convex functions, especially concave 
functions, in order to have a better approximation of ||x||o- In [31] Zhao and Li 
introduced the following function which is a combination of / p -norm(0 < p < 1) and 
the log function to approximate ||x|| ,(see Figure (2)): 

n n 

F e (x) = J2 MN + e ) + X^l^l + e ) P ' 0< p <1 - 




Figure 1: The graph of Yl7=i \ Xi \ P ^ or different values of < p < 1 . 




Figure 2: The graph of X/£=il°s(l a '*| + e ) + X^id^l + e ) P f° r different values of 
< p < 1. 



We will discuss this kind of functions later. Finite successive linear approximation 
algorithms have also been used and are still used to get an approximated solution of 
the concave approximation problems [2U [261 El 122] • 

Zi-minimization methods have been used to solve the problem fl2]). This is mo- 
tivated by the main idea of replacing ||x||o function with its local convex envelop, 
the Zi-norm function, and then solve the resulting linear program [SI [HI [5] . Under 
certain conditions the Zi-minimization method is able to obtain the exact solution of 
the problem (T5]) for very sparse solutions of the system Ax = b. In the literature, 
several conditions have been introduced and discussed for the equivalence between 
the Zi-minimization and the Zo- m i mm i z ation. The outstanding ones are spark [T8] , 
mutual coherence [HI [7J, restricted isometry property(RIP)[Sl HI [2], and null space 
property(NSP) [SI I]- For large optimization problems, the unconstrained version of 
the problem has been investigated in the literature, that may be referred as Lasso-type 
problems [30] . 

Numerical experiments show that weighted approaches are very affective in locat- 
ing an exact solution of the problem 021), and it can outperform other methods, in 
many situations [HI EEJ El EH E2l E] • 

Candes, Wakin, and Boyd [9] proposed a weighted /i-algorithm as follows: 

n 

Minimize > ujAxA 

tt (3) 

s.t. Ax = b. 

By introducing a diagonal matrix W = diag(u;i, 002, •••, u) n ), the problem above can be 
written as 

Minimize ||Wa;||i 

t A h (4) 

s.t. Ax = b. 

The weight can be interpreted as penalties for the components of the vector x. Larger 
penalties, (c^s, apply to smaller component of the vector x, for example one may 
choose the weights as t^t, i = l,...,n. However to avoid having infinity penalties, 
one may add a parameter like e > to define the following weight [9] 

u { = - — — — , i = l,...,n. 

\ x A + e 

Choosing a proper e to have a more efficient algorithm is one of the challenges. 
Very small or very large e might lead to improper weights which may cause the fail- 
ure of the algorithms. Since very small e might result in infinity penalties for small 
component of x, and with very big e the penalty might not recognize the difference 



between the small components of x and the large ones. We discuss the choice of e for 
our algorithms in the numerical experiment later. 

In an iterative reweighted Zi-algorithm, weights can be defined from the iteration 
in the previous step. Suppose the solution at the step I is x l , then the weight at the 
next step I + 1, can be given as u l+1 = , ;, +e . This was introduced by Candes, Wakin, 
and Boyd, and we refer to their algorithm as CWB, in this paper. 

In [31] , Zhao and Li introduced a unified framework for the reweighted /i-minimization. 
The main idea is to define a merit function which is a certain concave approximation 
of the cardinality function, and to construct different types of weights through the 
linearization techniques. Based on the class of merit functions defined by Zhao and 
Li [3l], we identify several new specific merit functions which are used to define the 
weights of the reweighted ^-algorithms. The main purpose of this paper is to study 
these merit functions, and to test the success probability of the reweighted algorithms 
associated with these merit functions for locating the sparsest solution of linear sys- 
tems, where the matrices, A, are generated based on different statistical distributions. 
Note that most of the previous experiments in the literature use normally distributed 
matrices. Also, we demonstrate how the parameters used in the algorithms may affect 
the performance of these methods. Furthermore, we evaluate what choice of e may 
make our algorithms work better. In section 2, we discuss different types of merit 
functions and the associated reweighted ^-algorithms. In section 3, we present and 
discuss our numerical results, and provide comparison between these algorithms. 

2 Merit functions and reweighted algorithms 

Merit functions have been used frequently in the field of optimization. Recently Zhao 
and Li [31] has used merit functions to approximate lo-novva. The merit function is 
defined as follows. 

Merit function: For any e > 0, a merit function F e (x) : K n — > M. for approx- 
imating the /cr n orm, is strictly concave, separable, coercive, strictly increasing and 
twice differentiable, with the following properties: 

1. lim e _> -4Q = \\x\\o + C, g(e) > is a function of e, and C is a constant, 

2. F e (x) =F e (\x\), Vi G I" , 

3. lim^. , e )^.(o,o) [VF e (i)]i = oo, Wx > 0, Ve > 0, 



4. lim e _ s> o[VF e (x)]j = q, Vxj > 0, where each q is a positive constant. 

After replacing the ||x||o by a merit function the problem ([2]) can be written as 

Minimize FAx) 

A (5) 

s.t. Ax = b. 

Note that F e (x) is a concave function. One of the usual methods to solve concave 
optimization problems is to apply the linearization method, which in this case is a 
special type of Majorization-Minimization(MM) method. For more illustration see 
that by applying the Taylor expansion of F e (x) around a point u, we conclude 

F e {x) < F e (u) + (VF e (u),x-u). 

The right hand side of the inequality above is a linear function. Hence the problem 
(JSJ) would be reduced to the following linear program: 

Minimize CVFJu),x) 

(6) 
s.t. Ax = b. 

So in our iterative reweighted algorithm, we solve at step / the following optimization 
problem: 

Minimize (VFJx l ),x) 
s.t. Ax = b, 

where x l is the solution of the previous iteration, and VF e (i') are the weights. The 
reweighted Zi-algorithm can be defined as follows: 

• Set I as an index which counts the iterations, and choose a small enough e > 0. 

• Step 0: Choose a starting point x 1 . This can be obtained by solving the l\- 
minimization problem. 

• Step /: Set u l = VF e (x l ), and Solve 

x l+l = argmin{(oj', x) : Ax = b}. (8) 

• Step I + 1: If some termination criteria holds, stop. Otherwise, set I 4— I + 1, 
and go to step /. 

An additional step can be added to the above algorithm concerning the choice of 
e. In this paper our updating rule is q + i = 0.5e«. In CWB algorithm, e is updated as 
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ei + i = max{|a;'|(i ), 0.001}, where zo = u lo m ( wyr , and \x\i is the biggest io elements of x. 

L 6\ m /J 

It is quit challenging to prove that under a mild condition, the reweighted l\- 
algorithm converges to the sparsest solution of problem flSJ). This is still an open 
questions in this field. However some progress have been made in this area [3H [231 
HHI3T]. Mangasarian [23] introduced a successive linearization algorithm(SLA) to find 
the solution of general complementarity problems, and proved that SLA algorithm 
terminates in finite number of iterations, and creates decreasing objective function 
values at each iteration. Furthermore he proved these values converge to a stationary 
point. Chen and Zhou in [H] proved that the sequence generated by reweighted 
/i-algorithm converges to a stationary point of a kind of truncated Z p -minimization 
problem (0 < p < 1). Similar results can also be found in [ST]. Recently, Zhao and Li 
[31] defined a range space property(RSP) for matrices, under which he proved that 
the reweighted ^-algorithm converges to certain sparse solution of the problem. 

Following the framework of the reweighted /i-algorithm in [31], we discuss some 
new merit functions. Before we go ahead, let's consider the following merit function 

n n 

F e (x) = Y, M W + e) + £(|*i| + eY, (9) 

i=\ i=l 



where < p < 1, which is mentioned in [M], based on which we will construct new 
merit functions. To verify that the above function is a merit function, one should 
check all the defined properties are satisfied. First, let's verify that this function is 
an approximation of Z -norm function. 

Indeed, it is easy to check that 

lim f _ EkMM±il±SkM±fn = ii^ii, 

6-*oV loge J 



Note that 



lim F e (x) = oo, 



which means that the function is coercive. It is clear that F e (x) = F t (\x\), and the 
function is increasing. In R 7 }, we have 



v ; , ( ; . , _ , '1 + (a* + efp 1 + (x n + efp 



, . . . , 



T 



X\ + e x n + e 

Also, for every i = l,...,n, we have 

1 -L (\ r .\ -U f)P n 

lim [VFMx\)]i= lim ' ;' '— = oo, i = l,...,n, 

(xi,£)->(0,0) (xi,e)->(0,0) \Xi\+e 



and VF e (x) is bounded when e — >■ 0. Since < p < 1 and x» > 0, we have 

p(xi + e) p > p 2 (xi + e) p , i = l, ..., n, 

so 

-1 + (xi + e)V - (xi + e) p p . 
■ < 0, i = l,...,n, 

Xi + e 

and hence 

V F e (x) = diag -< 0, i — l,...,n. 

V ^ + e / 

As seen, in f?" the Hessian of the above merit function is negative definite, so the 
function F e (x) is strictly concave. From the above discussion one can define the 
following weights for the reweighted /i-algorithm: 



i + (\ Xl \ + ey P . 

r^, » i = l,...,n. 



ut = [VF £ (\x\)]i 

\Xi\ + e 

Note that the item (2) of the definition of a merit function implies that \yF e (x l )]i — > 
oo as (x\, e) — > (0,0), which means larger penalties(weights) for the smaller elements 
of x, at each iteration. 

Now, we start to define a new merit function as follows 

n 

F e (x) = J2 Mlog(N + e + (\ Xi \ + ef)). (10) 

i=i 

We verify this function is a merit function. Clearly, this function is an approximation 
of / -norm function. Because 

log(log(xi + e + (xi + e) p )) 

hm — - — -—7 = 0, for Xi ^ 0, 

*-o log(log(e)) ir ' 

and 

log(log(x; + e + (x; + e) p )) 

hm = 1, for ^ = 0, 

z^o log(log(e)J 

we conclude that 

,. ZLi log(log(|^| + e + (|x,| + e)^)) 

hm : — t 1 — prr = n - \\x o- 

s ->° log(log(e)) 



In i?" , the gradient of F e (x) is given by 



(xj+e) p P 



[VF ( (x)]i ' + - r ' + ' 



(Xi + e + (^ + e)P)(log(xi + e + ( Xi + e)*»)) ' 
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and since \im x .^ Xi\og(xi) = 0, we have 



lim \VFJx)]i = oo. 

( a * I e)->(0,0) 



Note that for every xi > 0, 



lim[VF e (x)]i = ■ — p- — ■ — p- = a, i = l,..., n, 

e->0 " (Xi + X\) log(Xj + x£) 

where q is positive and bounded for every i = l,...,n. Also in i?™, V 2 F t (x) is a 
diagonal matrix with the following entries on its diagonal, 



[V 2 F £ (x)] ? 



(x t +e)V _ (x l +e)Pp 
(a=i+e) 2 (xi+e)2 



(a* + e + (x« + e)P) log(x, + e + (xi + e)?) 

2 



1 I (.Xj+e)Pp 
Xi+e 



(Xi + e + ( Xi + e)P) 2 logfxi + e + (x { + e)P) 



Xi+e 



(Xi + e + (xi + e)p) 2 log ( Xi + e + (x { + e)pf ' 



l,...,n. 



Since, for every i = 1, ..., n, [V 2 F e (x)]a < 0, we have 

V 2 F e (x) -< 0. 

So the function <Q is strictly concave, and it is a merit function. 

Based on OH]), the reweighted Zx-algorithm choose the following weights: 

I , {\xi\+e)Vp 
\xj\+e . _ , 

Ui " (\ Xi \ + e + (\ Xi \ + e)P)(log(|xi| + e + (\ Xi \ + e)P)) ' % " , '" ,U ' 

In this paper, we refer to W\ as the reweighted algorithm with the above weights. 
The Figure (15) shows the probability of success of W\ algorithm via different choices 
of e. This figure demonstrates that e = 0.01 works very good to locate the exact 
solution of the problem (j2]), when the sparsity is 15. Clearly the above weights are 
related to the parameter p, so we tested the performance of W\ algorithm for differ- 
ent sparsity of the solution, i.e, k = 5, 10, 15, 20, via different choices of p. Thirteen 
different values of p have been tested (matrix A has been normally distributed), and 
the result is summarized in Figure (14). Obviously the probability of success is higher 
when the sparsity of the solution is lower. This can be seen in Figure (14). 



Another new merit function can be defined as follows 



1 - 
F e (x) = -J2 Oog(N + e + (N + e) q )) p , (11) 



p '- 



where < p, q < 1, which is an approximation of ||x||o- In fact 

,. JItlOog(|^| +6+ (W +€)«))* m m 
n _ llm TTi 7 W5 = \\ x 0- 

In i?" , the gradient of F e (x) is given by 

log(x i + e+(x i + e)«) p (l + **±^ 



[v^Wli 



Xi+e 



(x, + e + (xi + e)«) log (xj + e + (x, + e) 



Note that lim( :Ci)£ )_ > .(o i o)[VF e (a;)]j = oo, and lim e _ > o[V-F e (a;)]j is bounded, for every fixed 
x > 0. In i?™, the Hessian is a diagonal matrix with the following diagonal elements 

[V 2 F e (*)] M - log(s, + 6 + te + e)«r 



(xi + e + (a* + e)«) log (xi + e + (x< + e)«) 

/ 



^-^(l + ^T?)' 



V 

(xj + e 



^ + e + ( Xi + e)«) log (xi + e + (a* + e) 9 ) 
Y - (Xi + e) q q \} + ~^+^ ) 



[Xi + er Xi + e + (x; 



+ 



l,...,n. 



/ 



where < p, q < 1. Clearly, V 2 F e (x) -< 0, which implies the function is strictly 
concave. Thus, the function ( ITTi) is a merit function, so we may choose the following 
weights in our algorithm: 

{\xi\+e)lq 



log(|z i | + e + (|a; i | + e)9) p (l + ^ 



Wi " (|x i | + e+(|x i | + e)9)log(|x i | + e+(|x i |+e)9)' * ~~ '•"■' ?2 ' 

We refer to W% as the reweighted algorithm with the weights above. The Figures 
(11), (12), (13) show the performance of Wi algorithm for finding the exact solution 
of the problem (T5]) for different choices of the weights parameters, p and q, and for 
different fixed sparsity of the solution, i.e., k — 5, 10, 15, 20. 

Remark.We see from above that the log function plays a vital rule in constructing 
a merit function. As pointed in [M| 133] , the log function can enhance the concavity 
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of a given function without affecting its coercivity and monotonicity. For the conver- 
gency analysis of the reweighted Zi-algorithms based on the class of merit functions 
that defined at the beginning of this chapter, one may refer to the Theorems 3.9 and 
3.11 in [3l], where it has been shown that under the so-called RSP condition, the 
algorithm may converge to a solution of problem (T5]) with certain level of sparsity. 



3 Numerical Experiments 

In this section, we compare the performance of the algorithms above for finding the 
exact solution of the problem (T5]) through the numerical tests. We compare the 
following algorithms in our numerical experiments. 
Zi-min: 



Minimize ||x||i 

s.t. Ax = b, 



(12) 



CWB(Candes, Wakin, Boyd): 



l+l ■ S~^ ■'■ i i 
x = argmm > — r . AxA 

tiK\ + ^ (13) 

s.t. Ax = b, 



W x : 



I | (H\+£ l ) p p 

l+l _ • V^ \x\\+e l , , 

, - argmm ^ (|^| + e i + pj + e ^) P )(l g(|^| + e l + {\x\\ + e 1 )?)) ^ (14) 
s.t. Ax = b, 



W 2 



j-irorniri ▼ — - — 

j^ (|x|| + e + {\x\\ + e l Y) log (|4| + e l + (\x\\ + e*)«) 



s.t. Ax = b, 



(15) 



where < p, q < 1, A e M 5UxiUU , b e M 5U , and x e 

In our numerical works, we randomly generated the matrix A e ]^50x2oo^ anc | f or 
a fixed sparsity, we randomly generated the solution vector x G M 200 . We tested 100 
randomly generated marices, A, for different level of fc-sparsity of the solution, i.e., 
k — 1,2, ...,26. The matrix A (the problem data) was randomly generated based on 
different statistical distributions. Most of the previous numerical experiments in the 
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literature usually use normally distributed matrices. 

The distributions that we considered were Normal (N(fi,a)) with the parameters 
/j, = and a — 1, Poisson (Pois(X)) with the parameter A = 2, Exponential (Exp(/j)) 
with the parameter n — 5, F-distribution (F(a,/3)) with the parameters a = 1 and 
/3 = 6, Gamma distribution (Gam(a,b)) with parameters a = 5 and b = 10, and 
Uniform distribution (U(N)) with the parameter iV = 10. The probability of success 
of the 4 algorithms mentioned above, i.e, /i-min, CWB, W\, W^ have been compared 
via different sparsity of the solution, and through all the above differently distributed 
matrices A. On a laptop with a Core 2 Duo CPU (2.00 GHz, 2.00GHz) and 4.00 
GB of RAM memory, each comparing figure took approximately 14-hours time (in 
average) . 

The updating rule e l+1 = \e l was used, at each iteration I. The choice of e is crucial 
for reweighted /i-algorithms. Hence, we have also tested the algorithms by applying 
Candes, Wakin, Boyd(CWB) updating rule for e, and also a fixed e = 0.01. These fig- 
ures demonstrate how these choices of e may affect the performance of the algorithms. 

As seen, the weights in W\ and Wi vary for different values of p and p, q. There- 
fore, we have tried different choices of p and q to find out how they may affect the 
success probability for W\ and Wi algorithms. 

In the Figure (3), the matrix A has been generated from Exp(/j), with \x — 5. 
We set p = 0.05 in W±, and p = q = 0.05 in Wi- As shown, all of the algorithms 
are very successful when ||x||o < 7. When 7 < ||x||o < 11, CWB, W\ and Wi almost 
perform the same as each other, but when ||x||o > 11, W\ and W<i outperform the 
CWB algorithm. All of the algorithms fail when the cardinality of the solution is 
above 25, i.e, ||x||o > 25. 

In Figure (4), the matrix A has been generated from Exp(fj,), with \x — 5, as in 
Figure (3). However, in this case we used different values for p and q. We chose 
p = q = 0.4, which is much larger than 0.05. As expected, both W\ and Wi perform 
significantly worse than the case of p = q = 0.05. Even for lower sparsity, both algo- 
rithms fail to locate the exact sparse solution with a high probability. 

In Figure (5), the matrix A has been generated from F(a,/3), with a = 1 and 
/3 — 6, and we set p = q = 0.05. As shown, all of the algorithms start failing when the 
cardinality of the solution is higher than 4, i.e, ||x||o > 4. W\ and W2 perform better 
than CWB for higher cardinality of the solution, and W2 is slightly better than W\ 
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in general. 

In Figure (6), the matrix A has been generated from Gam(a, b), with a = 5 and 
b = 10, and we set p = q = 0.05. For lower cardinality of the solution, CWB and 
W\ perform slightly better than Wi when ||x||o < 8. Also CWB, W\, and /i-min 
are completely successful for locating the exact solution, when ||x||o < 8. But for 
8 < || re || o < 10, only CWB and W\ are successful, /i-min, CWB, W 1 and W 2 fail 
when ||x||o > 17, ||x||o > 21, ||x||o > 24, ||x||o > 26, respectively. Therefore W\ and 
W2 perform significantly better for higher cardinality of the solution. 

In Figure (7), the matrix A has been generated from iV(/i,o"), with /i = and 
o = 1, and we set p = q = 0.05. As shown, /i-min, CWB, and W\ are very successful 
for finding the sparsest solution of the system when ||x||o < 8. W\ and W2 perform 
better than the other two algorithms for higher cardinality of the solution. 

In Figure (8), the matrix A has been generated from N(n, a), with /i = and a = 1 
as in the Figure(7). However, we chose bigger values for p and q, i.e, p = q = 0.4. For 
large values of p and q, W2 starts failing for ||x||o > 4, and performs much worst than 
W\, CWB, and /i-min. Also, for higher cardinality of the solution CWB performs 
better than W\ and Wi- Hence, from this figure and the Figure (4), one may conclude 
that smaller values for p and q should be chosen in order to achieve better results. 
Note that for large values of p and q the merit functions in W\ and W% are not good 
concave approximations of /o-norm. 

In Figure (9), the matrix A has been generated form U(N), with iV = 10, and 
we set p = q = 0.05. All of the algorithms except Wi are successful for finding the 
sparsest solution of the system when ||x||o < 9. For 9 < ||x||o < 12, CWB performs 
slightly better than W\ and W-i- But for higher cardinality of the solution, W\ and 
W2 outperform /i-min and CWB. 

In Figure(lO), the matrix A has been generated from Pois(X), with A = 5, and 
we set p = q = 0.05. All of the algorithms except W2 are successful for finding the 
sparsest solution of the system when ||x||o < 9. For higher cardinality of the solution, 
W\ and W2 outperform the other algorithms. 

Clearly, for small values of p, the best algorithm is W\ in general, i.e. for different 
cardinality of the solution and for different tested distributions. For all of the differ- 
ent tested distributions, both W\ and Wi (for small choices of p and q) outperform 
CWB when the cardinality of the solution is higher. 
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In the Figures (11), (12), (13), we focused on the performance of Wi algorithms 
for different values of p and q via different fixed cardinality of the solution. In the 
Figure (11), we fixed p = 0.08 and set different values of q. We examined the prob- 
ability of success of Wi for different fixed sparsity of 5,10,15,20. As expected, when 
cardinality of the solution is lower the success probability of Wi is higher. As seen, 
the probability of success for fixed sparsity of 5 is the highest, and the probability 
of success for fixed sparsity of 20 is the lowest. The Figures (12) and (13) show the 
same results for fixed p = 0.4 and p = 0.8, respectively. 

In the Figure (14), the performance of W\ has been tested using different choices of 
p and different fixed sparsity of the solution. As seen, in Figure (14), when p increases 
from 0.04 to 1, the probability of success of the algorithm becomes lower(except some 
jumps). As shown, for different fixed sparsity of 5,10,15,20 the highest probability of 
success was achieved when p = 0.04. Looking back to the merit function defined for 
the W\ algorithm, one may see that for smaller values of p the function is a better 
concave approximation of /o-norm. 

As we have discussed before, the choice of e for the reweighted /i-algorithm is 
important. Either very small or very big e may result in improper weights, which 
may cause the failure of the algorithms. In Figure (15), we fixed the sparsity of the 
solution (k = 15) and set p = 0.05. Different choices of e have been tested to suggest 
what e might be the good one for which W\ performs better. The matrix A has been 
generated from iV(0, 1). As shown, when e tends to zero (e.g. e ps 0.0001), or when e 
is big (e.g. e ~ 0.1), the probability of success decreases. Our numerical experiments, 
in Figure (15), show that e = 0.01 is a good choice for the weights in W\ algorithm. 

In Figure (16), we fixed e = 0.01 (with no updating rule) and compared the perfor- 
mance of /i-min, CWB, Wi, Wi. Like Figure (7), the matrix has been generated from 
N(0, 1), and we set p = q = 0.05. Our numerical experiment show that W\ and Wi 
significantly outperform CWB especially for higher cardinality of the solution. Com- 
paring Figure (16) and Figure (7), one may conclude that even for a fixed e, if chosen 
correctly, both W\ and Wi algorithms may perform very well to find a sparse solution. 

Again to show that how important the choice of e is, we compare the performance 
of Zi-min, CWB, W\, Wi based on the Candes updating rule. As seen in Figure(17), 
CWB outperforms both W\ and Wi for lower cardinality of the solution, i.e, when 
1 1 07 1| < 12 in our numerical experiments. 
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4 Conclusion 

We introduced a few concave approximations for the function ||x||o- These approx- 
imations can be employed to define new weights for the reweighted Zi-algorithms, 
which are used to locate the sparse solution of a linear system of equations. Through 
numerical experiments, we compared the performance of these reweighted algorithms 
and some existing reweighted algorithms when applied to linear systems with different 
statistically distributed matrices A, and with different sparsity of the solution. We 
have also explained when the new reweighted algorithms outperform some existing 
algorithms in different situations. We have also illustrated that how different choices 
of p and q may affect the performance of the algorithms. In addition, we have shown 
that the choices of e may remarkably affect the performance of these algorithms as 
well. 
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Figure 3: Comparing the performance of Zi-min, CWB, W%, W<i minimization via 
the probability of success for finding the exact A;-sparse solution of Ax = b, where 
A G M 50x200 ) b G M 50 , p = q = 0.05. Matrix A has been generated from Exponential 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k= 1,...,26. 
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Figure 4: Comparing the performance of Zi-min, CWB, W±, W% minimization via 
the probability of success for finding the exact A;-sparse solution of Ax = b, where 
A G R 50x200 5 b G M 50 , p = q = 0.4. Matrix A has been generated from Exponential 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k= 1,...,26. 
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Figure 5: Comparing the performance of Zi-min, CWB, W\, Wi minimization via the 
probability of success for finding the exact ^-sparse solution of Ax = b, where A G 
]R 50x200 , b e M 50 , p = q = 0.05. Matrix A has been generated from F-distribution. 100 
randomly generated matrices have been tested for different sparsity of k = 1, ..., 26. 




Figure 6: Comparing the performance of ^-min, CWB, W±, W<i minimization via 
the probability of success for finding the exact ^-sparse solution of Ax = b, where 
A E M 50x200 5 b e IR 50 , p = q = 0.05. Matrix A has been generated from Gamma 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k= 1,...,26. 
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Figure 7: Comparing the performance of Zi-min, CWB, W±, W2 minimization via 
the probability of success for finding the exact ^-sparse solution of Ax = b, where 
A G M 50x200 ) b G 1R 50 , p = q = 0.05. Matrix A has been generated from Normal 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k = 1,...,26. 






4 ,4 
• Ms 






Figure 8: Comparing the performance of Zi-min, CWB, W±, W% minimization via 
the probability of success for finding the exact ^-sparse solution of Ax = b, where 
A G R 50x200 5 b G 1R 50 , p = q = 0.4. Matrix A has been generated from Normal 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k= 1,...,26. 
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Figure 9: Comparing the performance of Zi-min, CWB, W±, W2 minimization via 
the probability of success for finding the exact ^-sparse solution of Ax = b, where 
A G M 50x200 ) b G M 50 , p = q = 0.05. Matrix A has been generated from Uniform 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k = 1,...,26. 




Figure 10: Comparing the performance of /x-min, CWB, Wi, Wi minimization via 
the probability of success for finding the exact ^-sparse solution of Ax = b, where 
A G M 50x200 5 b G M 50 , p = q = 0.05. Matrix A has been generated from Poisson 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k= 1,...,26. 
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Figure 11: Comparing the performance of Wi minimization for different q = 0.04 : 
0.08 : 1, p = 0.08 via the probability of success for finding the exact fc-sparse solution 
of Ax = b, where A e ]^50x2oo^ ^ ^ jg>50^ M a trix A has been generated from normal 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k = 5, 10, 15, 20. 
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Figure 12: Comparing the performance of Wi minimization for different q = 0.04 : 
0.08 : 1, p — 0.4 via the probability of success for finding the exact ^-sparse solution 
of Ax = b, where A e ]^50x2oo^ ^ ^ ]g>50^ M a t r ix A has been generated from normal 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k = 5, 10, 15, 20. 
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Figure 13: Comparing the performance of Wi minimization for different q = 0.04 : 
0.08 : 1, p = 0.8 via the probability of success for finding the exact k- sparse solution 
of Ax = b, where A e ]^50x2oo^ ^ ^ jg>50^ M a trix A has been generated from normal 
distribution. 100 randomly generated matrices have been tested for different sparsity 
of k = 5, 10, 15, 20. 
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Figure 14: Comparing the performance of W\ minimization for different p = 0.04 : 
0.08 : 1 via the probability of success for finding the exact ^-sparse solution of Ax = b, 
where A G ]j50x2oo^ ^ ^ IR 50 . Matrix A has been generated from normal distri- 
bution. 100 randomly generated matrices have been tested for different sparsity of 
fc = 5, 10, 15, 20. 
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Figure 15: Comparing the performance of W\ minimization using different e = 
0.00001,0.0001, 0.001,0.01,0.1 via the probability of success for finding the exact 
k = 15-sparse solution of Ax = b, where A G M 50x200 ; b G M 50 , p = 0.05. Matrix A 
has been generated from Normal distribution. 100 randomly generated matrices have 
been tested for different chosen epsilons. 
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Figure 16: Comparing the performance of Zi-min, CWB, Wi, W<i minimization using 
fixed e = 0.01 via the probability of success for finding the exact fc-sparse solution of 
Ax = b, where A G k 50x200 5 ft g ]^ 5 ° 5 p = q = 0.05. Matrix A has been generated from 
Normal distribution. 100 randomly generated matrices have been tested for different 
sparsity of k — 1, ..., 26. 
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Figure 17: Comparing the performance of /i-min, CWB, W\, W2 minimization using 
Candes updates rule via the probability of success for finding the exact fc-sparse 
solution of Ax = b, where A e M 50x200 , b G M 50 , p = q = 0.05. Matrix A has been 
generated from Normal distribution. 100 randomly generated matrices have been 
tested for different sparsity of k = 1, ...,26. 
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